Computing the Stationary Distribution Locally

Computing the stationary distribution of a large finite or countably infinite state space Markov Chain has become central to many problems such as statistical inference and network analysis. Standard methods involve large matrix multiplications as in power iteration, or simulations of long random walks, as in Markov Chain Monte Carlo (MCMC). For both methods, the convergence rate is is difficult to determine for general Markov chains. Power iteration is costly, as it is global and involves computation at every state. In this paper, we provide a novel local algorithm that answers whether a chosen state in a Markov chain has stationary probability larger than some ∆∈ (0,1), and outputs an estimate of the stationary probability for itself and other nearby states. Our algorithm runs in constant time with respect to the Markov chain, using information from a local neighborhood of the state on the graph induced by the Markov chain, which has constant size relative to the state space. The multiplicative error of the estimate is upper bounded by a function of the mixing properties of the Markov chain. Simulation results show Markov chains for which this method gives tight estimates.